Section: New Software and Platforms

museval

Keywords: Source Separation - Metric

Scientific Description: museval is a Python package aimed at evaluating audio source separation algorithm on the musdb corpus.

It as a scientific tool of high impact, but of limited transfer importance, since it is only (but widely) used by the community to evaluate performance in scientific publications.

Functional Description: The BSSEval metrics, as implemented in the [MATLAB toolboxes](http://bass-db.gforge.inria.fr/bss_eval/) and their re-implementation in [mir_eval](http://craffel.github.io/mir_eval/#module-mir_eval.separation) are widely used in the audio separation literature. One particularity of BSSEval is to compute the metrics after optimally matching the estimates to the true sources through linear distortion filters. This allows the criteria to be robust to some linear mismatches. Apart from the optional evaluation for all possible permutations of the sources, this matching is the reason for most of the computation cost of BSSEval, especially considering it is done for each evaluation window when the metrics are computed on a framewise basis.

For this package, we enabled the option of having _time invariant_ distortion filters, instead of necessarily taking them as varying over time as done in the previous versions of BSS eval. First, enabling this option _significantly reduces_ the computational cost for evaluation because matching needs to be done only once for the whole signal. Second, it introduces much more dynamics in the evaluation, because time-varying matching filters turn out to over-estimate performance. Third, this makes matching more robust, because true sources are not silent throughout the whole recording, while they often were for short windows.

Release Functional Description: This version makes museval compatible with the latest MUSDB package version

• Participant: Antoine Liutkus

• Contact: Antoine Liutkus

• Publication: The 2018 Signal Separation Evaluation Campaign